Skip to content

expand parser with live views, row expiry, posting indexes#28

Merged
emrberk merged 1 commit into
masterfrom
feat/v10-updates
Jul 23, 2026
Merged

expand parser with live views, row expiry, posting indexes#28
emrberk merged 1 commit into
masterfrom
feat/v10-updates

Conversation

@emrberk

@emrberk emrberk commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Brings the parser up to grammar parity with QuestDB OSS ≥ 9.4.0 and Enterprise ≥ 3.3.0, plus the two still-open feature PRs (LIVE VIEWS, mat-view row expiry). Every construct parses to a typed AST, round-trips through toSql(), and autocompletes. Also fills prior-minor gaps (9.3.0→9.4.0, 3.2.0→3.3.0) surfaced by a full audit, and improves autocomplete/highlighting for keywords the grammar reaches via the non-reserved identifier path.

Guiding rule: never reject syntax QuestDB accepts (extra leniency is fine).

What's been added

LIVE VIEWS — create/drop/alter, SHOW CREATE, and the ANCHOR window clause; live_views() function.

  • CREATE LIVE VIEW lv FLUSH EVERY 5s IN MEMORY 1h START FROM BEGINNING AS (SELECT * FROM t)
  • CREATE LIVE VIEW lv FLUSH EVERY 1s START FROM NOW AS SELECT ... FROM base
  • ALTER LIVE VIEW lv RESUME WAL FROM TXN 1, DROP LIVE VIEW IF EXISTS lv, SHOW CREATE LIVE VIEW lv
  • SELECT avg(x) OVER (ORDER BY ts ANCHOR EXPRESSION timestamp_floor('1d', ts)) FROM t
  • ... WINDOW w AS (ORDER BY ts ANCHOR DAILY '09:30')

Materialized-view EXPIRE ROWS row retention (WHEN / KEEP LATEST / KEEP [n] HIGHEST|LOWEST [CLEANUP EVERY]), plus ALTER … SET/DROP EXPIRE.

  • CREATE MATERIALIZED VIEW mv AS (SELECT * FROM base) EXPIRE ROWS WHEN v < 2.0 CLEANUP EVERY 30m
  • ... EXPIRE ROWS KEEP LATEST PARTITION BY sym
  • ALTER MATERIALIZED VIEW price_1h SET EXPIRE ROWS KEEP HIGHEST px PARTITION BY sym CLEANUP EVERY 1h
  • ALTER MATERIALIZED VIEW price_1h DROP EXPIRE ROWS

SAMPLE BY bare single-letter units ({n,U,T,s,m,h,d,w,M,y}, no leading digit).

  • SELECT ts, avg(x) FROM t SAMPLE BY w

REFRESH MATERIALIZED VIEW … STATS.

  • REFRESH MATERIALIZED VIEW mv STATS

REBASE WAL [INTO '<dir>'] on ALTER TABLE and ALTER MATERIALIZED VIEW.

  • ALTER TABLE t REBASE WAL INTO 'd1'
  • ALTER MATERIALIZED VIEW mv REBASE WAL

SHOW CREATE DATABASE [ (INCLUDE|EXCLUDE) (ALL | (cat, …)) ].

  • SHOW CREATE DATABASE INCLUDE (tables, views, users)
  • SHOW CREATE DATABASE EXCLUDE (materialized_views, service_accounts, acl)

Table FORMAT { PARQUET | NATIVE } on CREATE TABLE (3 positions) and ALTER TABLE … SET FORMAT.

  • CREATE TABLE t (a INT) TIMESTAMP(a) PARTITION BY DAY FORMAT PARQUET WAL
  • ALTER TABLE t SET FORMAT NATIVE

Enterprise SWITCH ROLE TO { PRIMARY | REPLICA } [TIMEOUT <ms>] / SWITCH STATUS.

  • SWITCH ROLE TO REPLICA TIMEOUT 5000
  • SWITCH STATUS

Column-level GRANT/REVOKE wildcards, 3-word permissions, and CONVERT PARTITION permission.

  • GRANT SELECT ON tab(* EXCLUDE(a, b)) TO alice
  • GRANT SET TABLE FORMAT ON t TO alice (also SET STORAGE POLICY / SET TABLE TYPE / REMOVE STORAGE POLICY)
  • GRANT CONVERT PARTITION TO PARQUET ON tab TO alice

Posting/bitmap indexesINDEX TYPE POSTING [DELTA|EF] [CAPACITY n] [INCLUDE(cols)] / TYPE BITMAP|NONE, column- and table-level, plus ALTER COLUMN … ADD INDEX.

  • CREATE TABLE t (s SYMBOL INDEX TYPE POSTING DELTA INCLUDE (a, b), a INT, ts TIMESTAMP) TIMESTAMP(ts)
  • CREATE TABLE x (t TIMESTAMP, s SYMBOL), INDEX(s TYPE POSTING) TIMESTAMP(t)
  • ALTER TABLE t ALTER COLUMN sym ADD INDEX TYPE POSTING INCLUDE (p)

Dynamic WINDOW JOIN bound — a column/cast/function expression as the RANGE BETWEEN … PRECEDING/FOLLOWING bound.

  • SELECT * FROM t WINDOW JOIN p ON t.sym = p.sym RANGE BETWEEN wndBound SECONDS PRECEDING AND wndBound SECONDS FOLLOWING
  • ... WINDOW JOIN prices p RANGE BETWEEN t.price::long minutes PRECEDING AND 1 minute FOLLOWING INCLUDE PREVAILING

Multi-RHS HORIZON JOIN chaining before a trailing RANGE/LIST … AS x.

  • SELECT ... FROM trades t HORIZON JOIN bids AS b ON (t.sym = b.sym) HORIZON JOIN asks AS a ON (t.sym = a.sym) LIST (-1s, 0s, 1s) AS h

Enterprise COPY PERMISSIONS FROM <src> TO <dst> and CONVERT PARTITION grant/revoke names.

  • COPY PERMISSIONS FROM src TO dst

New function names (autocomplete): sleep, regr_r2, is_end_of_month, kurtosis*, skewness*, array_agg, live_views, cume_dist, ntile, nth_value, backups.

Autocomplete & highlighting

  • New contextKeywordSuggestions re-surfaces keywords the grammar accepts only through the non-reserved identifier sub-rule (so Chevrotain reports them as the abstract IdentifierKeyword and they never appeared as hints): GRANT/REVOKE CONVERT PARTITION, SET TABLE FORMAT/TYPE, SET/REMOVE STORAGE POLICY, CONVERT PARTITION TO PARQUET|NATIVE, and SHOW CREATE DATABASE category names. Parsing already accepted these — this only affects the hints.
  • Registered now as a constant and the SHOW CREATE DATABASE category names (schema, views, acl, materialized_views, service_accounts) as non-reserved constants, so they classify/highlight consistently while staying usable as identifiers (START FROM NOW now highlights as a constant; now() stays a function).

Testing

  • New parse+AST and round-trip blocks in tests/parser.test.ts, autocomplete regression tests in tests/autocomplete.test.ts, and representative fixtures in tests/fixtures/docs-queries.json.
  • SQL for each feature was harvested from the corresponding QuestDB/Enterprise test suite and verified to parse here.
  • 6992 tests pass; typecheck, lint, and build are clean.

Note: no version bump / CHANGELOG release entry in this PR (functional changes only).

@emrberk
emrberk merged commit 6c1e477 into master Jul 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant